DOCUMENT RESUME 



ED 470 654 



CS 511 587 



AUTHOR 

TITLE 

INSTITUTION 

PUB DATE 
NOTE 
PUB TYPE 
EDRS PRICE 
DESCRIPTORS . 



IDENTIFIERS 



Metcalf, Kim K. ; Smith, Carl B.; Legan, Natalie A. 

Evaluation of National Geographic School Publishing 
Nonfiction Literacy Materials. Summary Report. 

ERIC Clearinghouse on Reading, English, and Communication, 
Bloomington, IN.; Indiana Univ., Bloomington. Indiana Center 
for Evaluation. 

2002-09-00 

26p. 

ERIC Publications (071) — Reports - Evaluative (142) 

EDRS Price MF01/PC02 Plus Postage. 

Data Collection; Elementary Education; Evaluation Methods; 
*Instructional Effectiveness; *Instructional Material 
Evaluation; * Instructional Materials; *Literacy; Nonfiction; 
*Reading Programs 
National Geographic Society 



ABSTRACT 

During the 2001-02 academic year, a purposive, national 
evaluation was undertaken of "Windows on Literacy" and "Reading Expeditions, " 
two new school-based programs produced by the School Publishing Division of 
the National Geographic Society (NGS) . The evaluation sought to determine the 
efficacy of the new materials for teachers and students. Questions posed were 
whether , student academic performance improved when they were engaged with 
these literacy materials; what were the most effective practices teachers 
used to improve student performance as the materials were used; and whether 
students' affective behavior changed as a result of their experiences with 
these materials. The sample reflected broad geographic diversity across the 
U.S., classes at each of Grades 1-6, and use of books in both "Windows on 
Literacy" and "Reading Expressions." Each participating teacher (n=24) agreed 
to use at least five books from the appropriate literacy series during the 
study, and each assisted in three data collection activities--student 
generated concept maps, a written teacher survey, and extended teacher ’ 
interviews by. telephone (during which they shed light on their practices when 
using the materials). Findings suggest that students' learning improved 
significantly with these literacy materials and that, teachers thought the 
books impacted their students' general' attitudes toward learning. Teachers 
also thought that the pictures and the format of the text enabled students to 
personally connect with the texts, thereby making them more receptive to the 
content. (NKA) 
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During the 2001-02 academic year, a purposive, national evaluation was undertaken of 
Windows on Literacy and Reading Expeditions, two new school-based programs produced 
by the School Publishing Division of The National Geographic Society (NGS). The 
evaluation joined the capacities of the ERIC Clearinghouse on Reading, English, and 
Communication Skills; the Family Learning Association; and the Indiana Center for 
Evaluation. The general intent of the evaluation was to determine the efficacy of the new 
materials for teachers and students. The present report summarizes the evaluation activities 
and findings associated with this project. 




Three questions guided the collection and analysis of date in the present project: 

Question ./, Does student academic performance improve when they are engaged with 
the literacy materials included in Windows on Literacy or Reading Expeditions! 



Question 2, What are the most effective practices used by teachers to improve student 
performance as the materials were used? 



Question 3, Does students’ affective behavior change as a result of their experience with 
the Windows on Literacy and Reading Expeditions materials? 



Question two above represents an attempt to describe how teachers used the literacy 
materials in their classrooms and, to the extent possible, to identify those practices or 
approaches that teachers felt were most useful to them. Questions one and three, in contrast, 
seek possible causal relationships between changes in students’ academic or affective 
performance that can be attributed to engagement with the materials. 

It was hoped that an experimental approach might be applied to the study of questions one 
and three. However, logistical and resource restrictions made this impossible. Instead, a 
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pretest-posttest design was applied to changes in students’ academic performance, and 
written and telephone surveys of teachers were used to estimate both changes in students’ 
affective behavior and to describe the instructional approaches used by teachers. These 
methodologies do not allow definitive conclusions to be drawn about the impact of the 
literacy materials on students. They do, however, provide an indication of changes in student 
performance that teachers believe to have resulted from engagement with the materials. 

2.1 Sample Selection 

Of fundamental importance to the evaluation team was that the sample reflected: (a) broad 
geographic diversity across the U.S.; (b) classes at each of grades 1-6; and (c) use of books in 
both Windows on Literacy and Reading Expeditions. Thus, in an attempt to identify 
potential study participants that would address these issues, evaluation team members 
worked with National Geographic staff in the late fall of 2001 to identify schools that had 
purchased the textbook series in recent months. Upon receiving contact information for these 
schools, evaluation staff members made initial contact with a representative of each building 
to identify specific teachers who were to be using the materials during spring semester, 2002. 

Individual teachers were then contacted by evaluation staff members who explained the study 
and asked each teacher if he or she would be interested in receiving detailed information 
related to the required research activities. Consenting teachers received a letter from the 
Indiana Center for Evaluation explaining the purpose of the study and detailing the three 
primary research activities (described below) in which each teacher would be required to 
participate. After letters were distributed, research staff members made telephone contact 
with teachers requesting additional information and answering teachers’ questions about the 
study. At the conclusion of this telephone conversation, research staff invited the 
participation of teachers. In all, 24 teachers were identified by the evaluation team and 
agreed to participate*. 



‘ Four teachers, who initially agreed to participate, dropped out of the study prior to returning any data. 
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2.2 Data Sources and Collection 

Each participating teacher agreed to use at least five books from the appropriate literacy 
series during the period of the saidy. The books and the order in which they were to be used 
were to be established by the teacher at the outset of the study and conveyed to the evaluation 
team. In addition, teachers agreed to assist the evaluation team by completing or 
participating in three data collection activities. These were: (i) student generated concept 
maps; (ii) a written survey of teachers; and (iii) extended telephone interviews with teachers. 
Each of these is described in detail below. 

2.2:1 Student Generated Concept Maps 

A particularly difficult element of the present project was the identification and collection of 
meaningful data on student learning across multiple classrooms, books, and book series. The 
literacy materials themselves are suitable for a variety of instructional approaches and could 
be used to promote a wide range of instructional outcomes. Thus, each teacher in the study 
could, potentially, have used the materials in unique ways and to foster learning outcomes 
that were specific to the students in his or her classroom. For example, some teachers might 
choose to use the materials to present content, others to supplement regular class content, 
others to motivate students to read on their own, and others to develop specific reading or 
literacy skills. To the extent this was the case, it would be extremely difficult to obtain 
measures of student learning through typical approaches (e.g., standardized tests). 

In order to allow collection of data associated with student learning that would be suitable 
across this range of applications (as well as across the content, levels, and specific skills 
addressed by individual books in the series), the evaluation team chose to use student- 
generated concept maps. The use of concept maps for collecting data is a technique that has 
been used frequently by the Indiana Center for Evaluation across a range of educational 
settings. ^ For the present study it was believed to be more efficacious than using other forms 
of assessment (e.g., multiple choice or open-ended tests) for three reasons. First, it provided 
a method of data collection that could be used across grades from first through sixth. 

^ See Kim K. Metcalf Evaluation of Project l.A.C.P. BridgCy technical report prepared for Junior Achievement 
of Central Texas (2001). 
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Second, the nature of this approach requires students not just to demonstrate knowledge of 
isolated facts, but also to demonstrate an understanding of the interrelationship between ideas 
or facts. Third, teachers’ use of concept maps has become much more prevalent in U.S. 
classrooms than traditional paper-and-pencil tests, thus, students would be more likely to 
have experience with the method of assessment to be used. 

Before engaging students in each of five books that the teacher selected from the NGS series, 
students were asked to complete a concept map of what they knew about the topic of the 
book. Because it was assumed that some students would be more experienced with concept 
maps than others, teachers were provided with standardized instructions that they could use 
to help students understand how to develop their maps. After working through each book, 
teachers again asked students to develop a concept map of what they knew about the topic of 
the book. 

Each pair (pre and post-reading) of student concept maps were stapled together by the 
teacher and returned to the evaluation team for scoring. Two teachers were unable to 
complete all five of the books they had selected. As a result, complete pre and post-reading 
data were available for only 18 of the 20 teachers. However, across the 20 classes from 
which one or more complete concept map pairs were obtained, 4,944 concept map pairs were 
available for analysis. 

Each student’s pair of maps (for each book) was scored by a trained member of the 
evaluation team for change in each of four factors: 

• Detail: This factor rated the extent to which a students’ post-reading concept map 
included greater detail about the core topic than the pretest. Ratings on this factor were 
from -2 (representing substantially less detail in the post-reading map) through 0 
(representing no change in detail) to +2 (representing substantially more detail in the post- 
reading map). 

• Abstraction: This factor rated the extent to which a student’s post-reading concept map 
included more levels of abstraction than the pre-reading map. As with detail, this factor 
was rated from -2 to +2. 
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• Number: This factor was a direct measure of change in the specific number of ideas, 
concepts, or facts (either abstract or detailed) that were included by the student from pre- 
reading to post-reading. 

. Overall: This factor represented a single rating of the scorer’s perception of the extent to 
which each set of concept maps indicated change in the student’s understanding of the 
topic. This factor was rated on a scale from -2 to +2. 

The reliability and consistency of the scoring process was established through random, 
independent checks of ratings by two other evaluation team members. 

2.2.2 Teacher Written Survey 

The second data collection activity was completion by each teacher of a brief written survey, 
which is included in Appendix A. The survey was mailed to each teacher at a point in the 
semester when they expected to be done with all five of the books they had selected. When 
all student concept maps had been received from a teacher, follow-up telephone and mail 
contact was made to remind them to complete and return the survey. All 20 teachers 
completed and returned the survey. 

The instrument asked teachers to provide or confirm information related to the grade or 
grades in which they used the books, the book series from which they selected, the titles of 
the five books they used for the study, the length of time during which they covered these 
books with their students, and the total number of books they had used.^ 

The largest portion of the survey was devoted to obtaining descriptive information about the 
literacy skills applied by their students. Teachers were asked to consider three distinct 
reading ability groups in their classes: the top 1/3, middle 1/3, and lowest 1/3. They were 
then asked to rate the extent to which students in each group used particular strategies when 
reading. These strategies reflected the literacy goals on which the NGS materials were 
focused (as they appeared in teacher materials), and related to strategies applied before 

^ Teachers were to use the five selected books (for the study) first. After this, they were free to use any other 
books from either series that they wished. 
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reading, during reading, and after reading. For each of 20 strategies, teachers rated each 
group of readers from “almost always” using the strategy to “almost never” using the 
strategy. Ratings were then coded from 1 (almost always) through 5 (almost never). 

2.2.3 Teacher Interviews 

The final data collection activity was conducted with each teacher after receipt of their 
written survey. Teachers were contacted via telephone and electronic mail to schedule a time 
convenient for conducting the interview. One member of the research team scheduled and 
conducted all of the interviews to ensure consistency in questioning. In all, 20 telephone 
interviews were completed, with interviews lasting 30 minutes on average. 

The open-ended instrument, consisting of a 15-question protocol, was designed by members 
of the research team to provide information regarding teacher’s perceptions and usage of the 
National Geographic textbook series. The protocol consisted of an opening paragraph, read 
by the interviewer, which identified the interviewer as an external evaluator who was not 
employed by National Geographic. The opening paragraph, in addition to describing the 
purpose of the interview, also informed the teachers that their response were confidential as 
their name would not be associated with their responses in any way. 

At the start of each interview, the researcher asked demographic questions regarding the 
context in which the reading activities were completed (e.g. grade level taught, book series 
used, and the order students experienced the books). This infonnation was used to update 
participant database information as appropriate. The next sections of questions focused on 
teachers’ perceptions of students’ interest and engagement with the materials, how teachers 
used the materials, and how the materials impacted the teacher’s instruction. To conclude 
each interview, feedback on the utility of the materials and how they might be improved was 
solicited. 

Specifically regarding student usage, teachers were asked the following questions: 

• Did you think the texts were leveled appropriately (appropriate for the age/ability 
they were intended)? 
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• In your opinion, what did your students like most or find most engaging about the 
books? 

• In your opinion, what disinterested or frustrated your students about the books? 

• How did the textbook series impact your students’ values and attitudes? 

Teachers were asked the following questions about their usage of the materials: 

• In what subject area did you utilize the National Geographic books? 

• Prior to using the National Geographic materials, what did you hope you and your 
students would gain from the materials? Were you satisfied? 

• Please describe the instructional/organizational strategies you used with the National 
Geographic materials? 

• How did the materials enable you to change your instruction? 

• As you were using the National Geographic materials, what pleased you most about 
the materials? (please describe) 

• As you were using the National Geographic materials, what most disappointed you 
about the materials? (please describe) 

And, lastly, to conclude the interview, teachers were asked to offer suggestions on how the 
materials might be improved: 

• If National Geographic asked your opinion about how they could improve the 
series, what suggestions would you offer? 

To document the telephone interviews, the researcher typed teachers’ responses verbatim, as 
the interviews were conducted. To ensure the accuracy of the transcriptions, upon the 
completion of every question, the researcher asked the teachers to review their responses as 
the interviewer read what she had typed. Teachers were invited to add to or modify their 
responses, and their additions/modifications were read back to them. This process continued 
until the teachers felt the transcription accurately represented what he/she intended to say. 
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M 

The present section of the report documents the data analysis techniques utilized throughout 
the evaluation of the National Geographic materials. Specifically, this section is organized 
according to data source. 




3.1 Data Analysis Techniques 

Concept Maps and Surveys 

Quantitative data from the concept maps and teacher survey were analyzed using both 
descriptive and inferential techniques. These included Chi-square analyses, analysis of 
variance, and when appropriate, follow-up pairwise comparison techniques."^ 

Interviews 

The extended telephone interviews provided qualitative data. These data were analyzed 
using Altas Ti © visual qualitative data analysis and management software. Interview 
responses were grouped according to question, and then one member of the research team 
coded and categorized teachers’ responses. 



4.0 Analyses 



The Analyses and Results section of this report is organized around the three questions that 
guided the evaluation of the National Geographic materials. For each question, the relevant 
results across data sources are presented. 



4 



An adaptation of the Dunn-Sidak (1977) multiple rejective technique was used for all follow-up comparisons. 
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4.1 Question One 



Does student academic performance improve when they are engaged with the literacy 
materials included in Windows on Literacy or Reading Expeditions! 



Research question one was addressed primarily through the pre and post-reading concept 
maps completed by students in each participating classroom and supplemented with data 
from the telephone interviews conducted with teachers. Specifically, the concept maps 
provide quantitative data regarding the knowledge gained by students as a result of their 
experiences with the materials, while the interviews and surveys provide information related 
to how teachers perceived the books to have impacted students. 

Across these data sources, the findings indicate that students’ learning improved 
significantly. Figures 1-4 graphically present data on each of the four elements measured 
through the concept maps. 

Figure 1. 




As reflected in Figure I , nearly 90% of all students demonstrated mean improvement in the 
amount of detail they included from pre to post-reading maps. Mean rating of improvement 
in amount of detail was 1.51 out of a possible 2.0. Surprisingly, only one (1) student in the 
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entire sample was found to have included less detail in post-reading maps, and only 4.2% 
exhibited no change. 

Figure 2. 




The level of abstraction evidenced by students was also found to be increased from pre to 
post-reading concept maps. While the extent of improvement in level of abstraction is not as 
great as that for level of detail (0.51 versus 1.51), mean abstraction scores indicated that 
86.3% of students demonstrated a higher level of abstraction in post-reading maps than in 
pre-reading maps. While slightly fewer than 14% of students demonstrated no change in 
level of abstraction, it is noteworthy that no students declined in their level of abstraction. 



Figure 3. 
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Figure 3 presents data on change from pre to post-reading in the total number of correct facts 
or ideas associated with the topic students included in their concept maps. Mean gain in 
number of items across students was 2.41 . Surprisingly, 95.3% of students included a greater 
number of correct ideas or facts in their post-reading maps. Of the remaining students, 4.2% 
demonstrated no significant increase and a single student demonstrated a slight mean decline. 



Figure 4. 




Across Figures 1-3, it is clear that students improved significantly and consistently in each of 
the three specific elements that were scored by the evaluation team. Perhaps equally 
important, however, are the ratings of overall quality of the concept maps. This rating would 
consider the totality of the map, its inclusiveness, accuracy, and detail. Figure 4 presents 
data associated with this more holistic measure of student learning as it is reflected in 
students’ concept maps. 

Mean rating of improvement in overall quality of learning reflected in students’ maps was 
1 . 13 out of a possible 2.0. Fully 96% of students were deemed to demonstrate greater 
understanding of the topic in their post-reading maps. Importantly, only 3.5% of students 
demonstrated no change in understanding and a single student demonstrated slightly less 
understanding. 
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As is evident across the measures of student learning that were examined through the concept 
maps, students’ demonstrated significantly greater understanding of the topics after working 
with the National Geographic materials. Surprisingly, this is true in the detail they were 
aware of, the level of abstraction with which they understood the various details, the number 
of correct facts or ideas that they understood, and in the accuracy with which they linked 
these elements. These quantitative findings are further supported by data drawn from the 
survey and interviews of teachers. 

Specifically regarding teachers’ perceptions of the impact of the National Geographic 
materials on their students’ academic performance, teachers identified increased acquisition 
of content knowledge and reading skills as areas in which they recognized student 
improvement. 

Content knowled2e 

One teacher talked about the content knowledge of two of her classes, she compared one 
class that experienced the National Geographic books with another that did not. She said. 

What I did notice was that my students understood the material better than my other 
classes [that did not use the National Geographic books] ...they performed much 
better on an end of the year test than the other kids... I think it was because they read 
and were exposed to the National Geographic materials where the other classes were 
not. 

A Title I reading teacher who used the series discussed how the books impacted her students. 
She explained, 

These kids were failing social studies...! matched the books with what they were 
learning in social studies, and they improved on their social studies class 
participation [because] it gave them some background knowledge. 
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Another teacher compared the information in the National Geographic materials with the 
information in the district adopted social studies textbook. “Often [in the National 
Geographic materials] there were specific examples that we had not heard about in our 
textbook,’* She explained, they offered “more realistic, accurate, and specific incidents that 
really brought this home for them,** identifying that because of the materials, her students 
understood that, 

there are people and real people who experienced all of this rather than the textbook 
that tends to be more dry and doesn V have all the people and personal experience. 
[The information] is told so realistically and so well. 

Reading skills 

Several teachers emphasized the use of the National Geographic materials in teaching their 
students reading skills. One teacher explained that she selected the National Geographic 
materials because, “I wanted [my students] to get experience with non-fiction and [to get] an 
understanding of how to find details and facts from texts.** Another teacher furthered this 
notion explaining, “You usually learn to read non-fiction when doing research and it is 
difficult... the [National Geographic] non-fiction text features are presented in such a way 
that they were perfect materials for kids learning to read non-fiction.” 

While the books teach social studies and science content, several of the teachers using the 
series were reading and language arts teachers. One of the reading teachers noted, 
“Obviously, I was not focusing on the content but was emphasizing the vocabulary and 
how to decode words, [but] the students obviously learned about the topic from reading the 
books.” She further explained, “because they were interested in the books... I used them to 
encourage and excite kids to figure out the words.” 

Another teacher identified how the structure and fonnat of the texts helped her struggling 
readers. They “liked the pictures and diagrams. [The pictures] really enhanced the texts [and 
they] helped lower readers because they rely on those cues more than higher readers do.’* 
Another teacher added, “they also found the glossary helpful, and I found it helpful too. They 
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used it when they didn’t understand the words in context of the book,” fiirther complimenting 
the structure and format of the National Geographic materials. 



4.2 Question Two 

What are the most effective practices used by teachers to improve student performance 
as the materials were used? 

Responses of teachers during the telephone interviews provide descriptive information 
related to the practices used by teachers as they were utilizing the National Geographic 
materials. 

One of the interview questions specifically asked teachers to describe how they utilized the 
materials in their classrooms. This question, in addition to responses to other questions, 
suggested that the National Geographic materials can easily be utilized for reading 
instruction and a means of integrating across the curriculum. Teachers also commented on 
the lessons provided in the series supporting materials, and how they used the materials to 
prepare their students for state proficiency tests, 

Reading Instruction 

One teacher explained how she used the books to help teach her students about textual 
features and how to use them. She explained, with the books I could “point out components 
of non-fiction reading and I could not do that with my units.” She explained, the National 
Geographic materials, 

gave me something written to work with.,. [I] gave them the books [and] described 
the bold print for use of special vocabulary words, [components] in non-fiction 
reading [like] titles on each page, [and] underlined words. 
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Another teacher explained how she used the materials to teach her students expository 
writing; she said “kids have trouble getting away from report like talk, [the materials serve as 
an] excellent example of how to write non-fiction/' 

Yet another teacher described how she uses the National Geographic materials in her 
instruction, “Well, they allowed me to work with different levels of readers.... the range of 
the books allows me to work with a variety of readers at their level.” She continued by 
explaining “I could use phonics skills or comprehensions skills with the books” to help kids 
to improve their reading. 

Curriculum integration 

Several of the teachers noted that the materials and the diversity of topics enabled them to 
integrate across the curriculum more easily. As one teacher explained, they “enabled me to 
better integrate instruction with social studies [by allowing] me to go beyond the textbooks 
[to] go outside of the social studies textbooks.” She explained, “during EL A 
[English/Language Arts] time, instead of reading fairy tales, I could integrate [with] social 
studies.” 

Another teacher explained, “we wish more textbooks would do what National Geographic 
did.” The books give “me exactly what I needed as opposed to instruction that is hit or miss. 
[For example] maybe you have a story that has point of view, but isn’t on target [for a child’s 
ability], these books are on target for these [skills] exactly.” 

Supporting materials 

Teachers who had the teacher resource guides commended National Geographic on the 
quality and quantity of the materials included. “Amazing materials that were included,” one 
teacher declared; “The information and primary source documentation was fantastic...! 
would find myself so interested in learning things I didn’t know that it surprised me.” 
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As another teacher explained, my favorite part of the materials was “probably the activities 
they provided. They were good because I could plan lessons around them, so they took some 
work out of the process of creating lessons.” 

The fact “that [the lessons in the teacher’s guide] were pretty easy once I explained them to 
kids” was most pleasing to one teacher. She explained “the suggested activities worked well 
with other activities the kids had done in school, so they understood them” with little 
additional instruction. While another teacher, when talking about the teacher resource 
materials said “I found the website accessible to find infonnation, and I used that a lot.” 
Several teachers mentioned using the National Geographic website to obtain additional 
information and/or ideas for using the books in their classrooms. 

When asked about her favorite feature of the materials, another explained, 

I liked the lay out; it is kid friendly. They are not like a chapter book, so the kids don 7 
have to plow through them without any pictures [and] they are not like a textbook 
with tons of words. [With textbooks] kids see so many words and they click off I like 
the thin books. The chapters are not overwhelming, and the facts on the pages were 
interesting. [The kids liked the identification of] websites they could go to, and we 
would go to them and check things out. 

State Proficiency tests 

Interestingly, a few teachers in the study discussed the connection between the curriculum in 
National Geographic materials and their state district curriculum standards. A teacher from 
Ohio stated. 

Before we saw the materials, we were just pulling things trying to reinforce the OPT 
[Ohio Proficiency Test] strands. When I saw how they [the National Geographic 
materials] matched the proficiency tests, I saw how they could master the [OPT] 
strands like cause and effect. 
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She identified herself as sold on the materials when she recognized that “every single book in 
the series is aligned with OPT strands,” and expressed relief at finding one source to 
reinforce the state test strands. 

Another teacher explained, 

I liked the format and the set up. Each page had a lot of graphs and tables, which is 
very good because our fourth grade state test [in Illinois] covers social studies and 
science. There are a lot of tables and graphs on the test, and any extra practice they 
can get is great. 

4,3 Question Three 

Does students’ affective behavior change as a result of their experience with the 
Windows on Literacy and Reading Expeditions materials? 

Question three is addressed through responses of teachers to the interview questions. 

Through the interviews, teachers identified several ways in which the materials impacted 
their students’ affective behaviors. Specifically, teachers thought the books impacted their 
students’ general attitudes toward learning, and more specifically their students’ interest in 
the content areas of science and social studies. Teachers also identified that they thought the 
pictures and the format of the text enabled the students to personally connect with the texts 
thereby making them more receptive to the content. 

Attitudes toward learning 



One teacher described that her students “realized that they learned things from the 
books., .and it inspired them to learn more through personal exploration.” She explained that 
her students used the books as starting points for topics that they further explored in 
independent study units. 
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Another teacher explained, the kids “they took pride in the books and they were anxious to 
read the next book. Often they would read ahead to see what would happen next.” The books 
engaged the kids. 



Interest in content areas of science and social studies 

Several teachers mentioned the impact the books had on their students’ interest in science 
and social studies. As one teacher identified, 

/ think the books improved their attitudes about studying social studies because [the 
topics in the books] were more relevant to their everyday lives. They learned things 
they could uses in their lives, particularly [the books] about managing money and 
being consumers. 

While another teacher added, the books affected student’s “attitude, they made them hungrier 
for information. They looked up things on their own and their attitude became more positive 
about doing research.” “The content was interesting,” identified another teacher, “it 
intrigued even my reluctant readers.” 

The “Kids Make a Difference” series of books was a focus for one teacher. 

I think the Kids series really hit home because they could relate to it We made up 
budgets, examined advertising, and looked at ads in kids ' magazines. They looked at 
the magazine and counted the number of ads in a 95 page magazine and were like 
'wow we are paying a lot for 45 pages of ads, ’ It hit home more with them because 
they could relate to it I think it may have impacted their values and some of them 
may go home and help their parents make better decisions like buying things that they 
get more for their money. 
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Personal Connection 



Many teachers emphasized the personal connection the students could make with the texts as 
a result of the format and writing style of the books, and they identified the impact the 
personal connections had on their learning. Related to the content of the books, one teacher 
explained, 

Well, I think it reinforced a lot of what they already know and made them feel 
important. They were not intimidated by it; they could relate to it so it made it more 
user-friendly to teachers as well as student. This made it interesting to me even. 

“I think what was nice was that our textbook does not have a lot of first person narrative type 
writing in it,” explained another teacher. The textbooks are 

good about showing real photos of that time, but it isn 7 enough. The National 
Geographic books were nice because they had art of the time, and art trying to depict 
photos of real people of the time. The characters were telling feelings and stories of 
people of the time and contrasting that with another s feelings. This allowed us to 
have more ‘what-if discussions. 

She continued, 

Nowhere in textbooks is there talking by a character. Even if [the characters in the 
National Geographic books] were invented, the kids believed it and it allowed us to 
talk about putting ourselves in that situation. It made history more real for them. The 
pictures were awesome, the subjects really appealed to them because they could 
relate them to their lives, and they were really excited to find out about new topics. 



During the telephone interviews, several teachers offered suggestions on ways National 
Geographic could improve the books and/or the series in general. Study participants offered 
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suggestions ranging from including more books in the series, to mapping the books to 
curriculums, to providing more supplemental materials, to offering services to teachers. 
Responses of teacher are organized within these categories and are included below. 



More books 



Several teachers mentioned a desire for more books in the series. One teacher explained, it 

would have been nice to have books covering the same content written at a variety of 
reading levels. For example, having three books, each written at different reading 
levels that covered the topics in say the ''Blue and Grey ” book. 

While another teacher offered the idea of expanding the number of books to cover topics 
more thoroughly; she offered perhaps having 

follow-up texts on the subjects that would go into the topics deeper.. ..so like have a 
few books related to electricity, including one that is general and then a few others 
that go more deeper into the issues. 



Yet another teacher suggested adding “good science books for the older kids would be 
great,” indicating “There are not a lot of fourth through sixth grade reading leveled books 
with a science focus.” 



Book content mapped to curriculum 



While a few of the teachers (e.g. those in Illinois and Ohio) specifically identified how the 
books directly mapped to their state curriculum standards, some teachers from other states 
expressed interest in books that were more directly mapped to their curriculum. 
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As one teacher expressed, “Some of the books at the fourth grade level worked with the fifth 
grade curriculum better than the fourth grade curriculum,” while the books that fit our fourth 
grade world history curriculum were targeted for third grade readers. Another teacher, one 
from New York, explained a similar situation; 

1 would like to see more books about land forms [for third graders]. We [ in New 
York] look at the globe and break it to continents and areas, for example, mountain 
range, polar, rainforest, grasses. 

She suggests “linking the book topics to state curriculum at the grade level the books are 
targeted towards.” 



Supplemental materials 



Suggestions for adding supplemental materials to the books were offered by many teachers. 
Some of the teachers making the suggestions indicated they did not have the teacher resource 
guides that corresponded with the texts, or that they were sharing one with several other 
teachers and so didn’t get to use it often. As a result, some of the ideas expressed may be 
addressed currently within the teacher resource guides unbeknownst to them. 



“Adding some questions to go with the books,” was a suggestions one teacher offered. While 
another suggested “in student copies, have some comprehension checks along the way.. . [or] 
give some writing prompts.” 



Other teachers offered ideas of additional materials to include as teaching resources in 
addition to the books themselves. “I would like to see,” explained one teacher, 

books that followed the series or a video, [or perhaps] a regular reading book [like a 
novel] that was linked to what they were doing. [For example a book] isolating one 
inventor and talking about that alone. 
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Another teacher suggested having some videos to go with the books and thought, 

maybe they could include a little diary or journal entry book, [National Geographic 
could ] print up a little diary or dialogue journal, maybe 10 pages, that you could 
order in sets of 10, so [teachers] could have these sets of pre-done booklets that 
match back to the books thdt have writing prompts in them. 



Services to teachers 

Three teachers specifically identified some ways they thought National Geographic could 
provide more services to the teachers who buy the books. One teacher explained, when I 
received the books, 

I didn 7 know for sure which ones were science or social studies titles. I had a hard 
time knowing where to put what book and how to organize them, and [to know] ifTd 
received everything I ordered. 

She suggested including an “instruction sheet... when you first get your crate... telling you 
which titles are social studies and which are science [and offering] a list of the levels of the 
books.” She explained having this information would enable her to know how to arrange her 
materials better. 



Two other teachers suggested allowing “teachers to share the materials they have used with 
the various books,” one of whom offered, perhaps having a chat site or “space on the web- 
site for teachers to offer ideas for how they have used the materials in their classroom and to 
provide activity or extension ideas.” 
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The present study was intended to address three questions associated with the effectiveness 
and utility of Windows on Literacy and Reading Expeditions, recently released non-fiction 
series for children. Drawing from a sample of teachers in grades I -6 throughout the country, 
data were collected on the impact of the materials on students’ learning and affect, the 
instructional approaches used by teachers as they worked with their children through the 
materials, and teachers’ perceptions of the materials themselves. 

Across the multiple data collection and analysis techniques used in the project, the findings 
consistently and clearly indicate that students whose teacher used the National Geographic 
materials demonstrated much greater and more complete understanding of the topics 
presented in the materials. Students were much better able to demonstrate broader, more 
detailed, and more integrated understanding of the topics after they had participated in their 
classes’ activities. 

Further, teachers were unanimous in their perceptions of the high quality and usefulness of 
the materials. They believed that the materials were particularly effective in promoting 
students’ learning of the science and social studies concepts presented in the materials, that 
the format and structure of the materials were engaging for students, and that the materials 
were of greatest benefit to students whose reading abilities were somewhat low. Because the 
materials were interesting and engaging for students, teachers also found them to be useful 
tools for developing reading and literacy skills among their students. 

Clearly, using the materials was believed by teachers to be beneficial for their students and 
data on student learning support this. These data are drawn from a study that employed a 
non-experimental design. Thus, the specific magnitude of the effects of the program on 
student learning cannot be estimated with certainty. It is recommended that a study using an 
experimental design be implemented to reliably confirm the impact of the materials on 
students and teachers. 
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The present findings are encouraging. The data obtained in the present study suggest that the 
National Geographic series Windows on Literacy and Reading Expeditions are extremely 
effective for both students and teachers. 
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